Repository navigation
docs: correct Playwright option support for Browser checks - #821
Conversation
- Mark channel as supported and explain how to use Google Chrome, which the Browser checks overview already lists. - Explain that launchOptions are ignored, and which launch options and Chromium arguments chromium.launch() keeps. - Mark deviceScaleFactor, testIdAttribute, contextOptions and bypassCSP as global configuration only, and list what test.use() accepts. - Add the missing built-in Node.js modules and the node: prefix to the runtime specification. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
hlenke
left a comment
There was a problem hiding this comment.
Ready to merge · The option tables, the test.use() allow-list, the launch-option and Chromium-argument allow-lists, the log lines and the module list all match the Browser check runtime on monorepo main. The inline comments are optional wording.
CI is green and the PR merges cleanly. #813 also edits playwright-support.mdx, but only the video sentence, which this PR does not touch, so either can land first.
Inline comments: 3 wording (all optional).
What I checked
- Read the whole diff and the full PR versions of
playwright-support.mdxand the built-in module section ofruntime-specification.mdx(merge basee0e0277f; nothing onmainsince touches these files). test.use()allow-list (the 20 options listed): identical to the allowed local-configuration keys in the Browser check runtime's Playwright Test mock (packages/check-runtimes-common/src/getLightweightPlaywrightTestMock.js). Any other key is dropped and logsLocal configuration key "<key>" is not supported by Checkly., as the PR says. The newer runner logs the same line.launchOptionsignored in the global configuration: the generated Playwright config writes Checkly's ownlaunchOptionsover the user'suse(getPlaywrightTestConfig.js).chromium.launch()keeps onlychannel,logger,proxy,slowMo,timeout; always headless; only--disable-web-securityand--host-resolver-rules=…args kept; warning text: all matchpackages/check-runtimes-common/src/browser/getPatchedChromiumProxy.jsand its copy in the newer runner. The warning in the PR is the runtime's message verbatim.channel: 'chrome'must be a literal string: the check scheduler picks a runner with Chrome by matching that literal in the check's files, including dependencies andchromium.launch({ channel: 'chrome' }). The runtime logs that the check falls back to Chromium when it runs on a runner without Chrome. That is the basis for the inline suggestion on line 143.- Built-in modules (
dgram,https,os,perf_hooks,stream/promises, and thenode:prefix): match the module allow-list inpackages/check-runtimes-common/src/runtimes/runtime-constants.js, which adds every module with and withoutnode:. gh pr checks 821: all pass.gh pr view: MERGEABLE / CLEAN.scripts/pr_impact.py: only an info note that the Browser checks overview is a hub page; the link anchors#use-google-chromeand#browser-launch-optionsexist on the PR branch.- End-to-end: not run against local Checkly. Browser check runners are not part of the local app, so I verified against runtime source. The PR description lists runs through the Browser check runtime.
- Could not verify: the existing
fs (partial…)line, which is not on the PR's lines.
| | `trace` | ❌ | | ||
| | `video` | ❌ | | ||
|
|
||
| In a test file, `test.use()` accepts only these options: `acceptDownloads`, `actionTimeout`, `baseURL`, `channel`, `colorScheme`, `extraHTTPHeaders`, `geolocation`, `hasTouch`, `httpCredentials`, `ignoreHTTPSErrors`, `isMobile`, `javaScriptEnabled`, `locale`, `navigationTimeout`, `offline`, `permissions`, `proxy`, `timezoneId`, `userAgent` and `viewport`. Checkly ignores any other option in `test.use()` and logs a line like this one: |
There was a problem hiding this comment.
Wording (optional): this list includes acceptDownloads and userAgent, but the Use table above has no rows for them. A reader scanning the table would think they're unsupported. Adding two ✅ rows keeps the table and this list in agreement.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hlenke
left a comment
There was a problem hiding this comment.
Ready to merge · The three commits since my last review (a7709db2..5b5deca4) apply two of my suggestions and add the runtime 2026.04 notes for dns, os and perf_hooks, which match the sandbox replacements on monorepo main. One optional item is left from the last review.
This is an incremental review of a7709db2..5b5deca4. No merge commits in that range. CI is green and the PR merges cleanly. Since the merge base, main changed playwright-support.mdx once, with #813 (the video sentence). GitHub still reports no conflict.
Inline comments: 1 wording (optional).
What I checked
- Read the full diff
a7709db2..5b5deca4(3 files) and the whole PR version ofplaywright-support.mdx. Read the built-in module list inruntime-specification.mdx. dnsin2026.04: on monorepomain, the dns replacement keeps the servers and the result order per check sandbox.setServers()andsetDefaultResultOrder()never change the runner process resolver (packages/check-runtimes-common/src/vm/sandboxedBuiltins.js). Matches line 50.osin2026.04: the replacement makes everyosfunction throwos.<name>() is not available in the Checkly check sandbox. It then restoresplatform(),arch(),type(),endianness(),tmpdir()andhomedir(), plusEOL,devNullandconstants. That is the exact list on line 53.perf_hooksin2026.04: the replacement exposes only a per-checkperformanceobject. Matches line 55.- Only
2026.04: the replacements are enabled only when the runtime version equals2026.04(packages/check-runtimes-common/src/doesRuntimeUseVm2v312.js). That backs "in runtime2026.04" on all three lines. - The two applied suggestions (overview Chrome bullet, the literal-string fallback sentence) are the text from my last review.
gh pr checks 821: all pass.gh pr view: MERGEABLE / CLEAN.git log e0e0277f..origin/mainon the touched files: only #813 (cdb4655a).scripts/pr_impact.py: only the info note that the Browser checks overview is a hub page.- End-to-end: not run. Browser check runners are not part of the local app, so I checked these claims against the runtime source (monorepo
deb338ba).
| @@ -87,20 +87,28 @@ We currently support the following configuration options. We will update this li | |||
| | `offline` | ✅ | | |||
| | `actionTimeout` | ✅ | | |||
| | `navigationTimeout ` | ✅ | | |||
There was a problem hiding this comment.
Wording (optional): you answered "add these" on my last review, but the Use table still has no acceptDownloads or userAgent row, although the test.use() list below includes both. This adds the two rows:
| | `navigationTimeout ` | ✅ | | |
| | `navigationTimeout ` | ✅ | | |
| | `acceptDownloads` | ✅ | | |
| | `userAgent` | ✅ | |
What changed and why
The Playwright support page for Browser checks disagreed with how Browser checks behave:
channelas unsupported, while the Browser checks overview lists Google Chrome as a supported browser.channel: 'chrome'works, so the row is now ✅ and a short Use Google Chrome section shows how. The overview links to it.launchOptionsas unsupported without saying what happens when you launch a browser yourself. A new Browser launch options section lists the launch options and Chromium argumentschromium.launch()keeps, and the warning logged for dropped arguments.testIdAttributeanddeviceScaleFactor(andcontextOptions,bypassCSP) as supported, but they only work in the global configuration:test.use()drops them and logs a line. Those rows now say "Global configuration only", and a paragraph lists the optionstest.use()accepts.dgram,https,os,perf_hooksandstream/promises, and didn't mention thenode:prefix. In runtime 2026.04,osandperf_hooksare now marked as partial and thednssettings as applying to one check, matching the sandbox replacements that runtime uses since vm2 3.12.The edits stay out of the lines #813 changes (video sentence) and don't touch the troubleshooting page (#818).
How each claim was verified
Browser check runners can't run in a local dev stack, so the claims were tested by running real check scripts through the production Browser check runtime for runtime 2026.04 (the vm2 sandbox, the Playwright Test mock and the patched browser launcher from the runner code at monorepo main
84921d2637), with the Playwright config the runner generates for scheduled runs, against real Chromium and Google Chrome.test.use({ channel: 'chrome' })runs the check in Google ChromeChrome/156…from the Google Chrome binary instead ofHeadlessChrome/145…chromium.launch({ channel: 'chrome' })also starts Chromechannel: 'chrome'literally in the codelaunchOptionsin the global configuration is ignoredplaywrightConfig.use.launchOptionsnever reached the browser command line; no loglaunchOptionsintest.use()is ignored and loggedLocal configuration key "launchOptions" is not supported by Checkly.chromium.launch()keepschannel,logger,proxy,slowMo,timeoutERR_PROXY_CONNECTION_FAILED,slowMo: 400slowed 5 clicks from 0.6 s to 2.2 s,timeout: 1failed the launch)headless: false,devtools: true,executablePath: '/nonexistent/chrome',ignoreDefaultArgs: truehad no effect; command line kept--headless--disable-web-securityand--host-resolver-rules=...are keptMAPrule resolved a test host;--use-fake-device-for-media-stream,--enable-webgl,--window-sizedroppeddeviceScaleFactor,testIdAttribute,contextOptions,bypassCSPwork in the global configurationgetByTestIduseddata-qa, reduced motion on, inline script ran despite a CSPtest.use()with a log lineLocal configuration key "<key>" is not supported by Checkly.line eachtest.use()allow-listscreen,defaultBrowserType,storageState,video,trace,headless,browserName,serviceWorkerswere each logged and droppeddgram,https,os,perf_hooks,stream/promises, and thenode:prefixnode:; modules outside the list (e.g.child_process,net,http2) failed withCannot find module. The monorepo's runtime harness golden files show the same modules loading in every runtime from 2022.10 to 2026.04Not tested / left out
channelin the global configuration, requesting an exception for a dropped Chromium argument, and listinghttpas a built-in module (available in Browser checks, not in API check scripts).🤖 Generated with Claude Code